Boosting Bayesian Parameter Inference of Nonlinear Stochastic Differential Equation Models by Hamiltonian Scale Separation

Parameter inference is a fundamental problem in data-driven modeling. Given observed data that is believed to be a realization of some parameterized model, the aim is to find parameter values that are able to explain the observed data. In many situations, the dominant sources of uncertainty must be included into the model, for making reliable predictions. This naturally leads to stochastic models. Stochastic models render parameter inference much harder, as the aim then is to find a distribution of likely parameter values. In Bayesian statistics, which is a consistent framework for data-driven learning, this so-called posterior distribution can be used to make probabilistic predictions. We propose a novel, exact and very efficient approach for generating posterior parameter distributions, for stochastic differential equation models calibrated to measured time-series. The algorithm is inspired by re-interpreting the posterior distribution as a statistical mechanics partition function of an object akin to a polymer, where the measurements are mapped on heavier beads compared to those of the simulated data. To arrive at distribution samples, we employ a Hamiltonian Monte Carlo approach combined with a multiple time-scale integration. A separation of time scales naturally arises if either the number of measurement points or the number of simulation points becomes large. Furthermore, at least for 1D problems, we can decouple the harmonic modes between measurement points and solve the fastest part of their dynamics analytically. Our approach is applicable to a wide range of inference problems and is highly parallelizable.Comment: 15 pages, 8 figure

Bayesian parameter inference with stochastic solar dynamo models

Time-series of cosmogenic radionuclides stored in natural archives such as ice cores and tree rings are a proxy for solar magnetic activity on multi-millennial time-scales. Radionuclides data exhibit a number of interesting features such as intermittent stable cycles of high periods and Grand Minima. Although a lot of effort has gone into the development of sound physically based stochastic solar dynamo models, it is still largely unclear what are the underlying mechanisms that lead to the observed phenomena. Answering these questions requires quantitatively calibrating the models to the data and comparing performances of different models with the associated uncertainties in model parameters and predictions. Bayesian statistics is a consistent framework for parameter inference where knowledge about model parameters is expressed through probability distributions and updated using measured data. However, Bayesian inference with non-linear stochastic models can become computationally extremely expensive and it is therefore hardly ever applied. In recent years, sophisticated and scalable algorithms have emerged, which have the potential of making Bayesian inference for complex stochastic models feasible. We intend to investigate the power of Approximate Bayesian Computation (ABC) and Hamiltonian Monte Carlo (HMC) algorithms. We present our first inference results with stochastic solar dynamo models

New Tools in Bio-NMR: Indirect Detection of Nitrogen-14 in Solids

This paper presents an overview of recently developed methods for the indirect detection of N-14 nuclei (spin I = 1) in spinning solids by nuclear magnetic resonance spectroscopy. These methods exploit the transfer of coherence from a neighboring 'spy' nucleus with spin S = (1)/(2), such as 130 or H-1, to single- or double-quantum transitions of 14N nuclei. The two-dimensional correlation methods presented here are closely related to the well-known heteronuclear single- and multiple-quantum correlation (HSQC and HMQC, respectively) experiments, already widely used for the investigation of molecules in liquids. Nitrogen-14 NMR spectra exhibit powder patterns characterized by second- and third-order quadrupolar couplings which can provide important information about structure and dynamics of molecules in powder samples

Accelerating phylogeny-aware alignment with indel evolution using short time Fourier transform

Recently we presented a frequentist dynamic pro- gramming (DP) approach for multiple sequence alignment based on the explicit model of indel evolution Poisson Indel Process (PIP). This phylogeny-aware approach produces evolutionary meaningful gap patterns and is robust to the ‘over-alignment’ bias. Despite linear time complexity for the computation of marginal likelihoods, the overall method’s complexity is cubic in sequence length. Inspired by the popular aligner MAFFT, we propose a new technique to accelerate the evolutionary indel based alignment. Amino acid sequences are converted to sequences representing their physicochemical properties, and homologous blocks are identified by multi-scale short-time Fourier transform. Three three-dimensional DP matrices are then created under PIP, with homologous blocks defining sparse structures where most cells are excluded from the calculations. The homologous blocks are connected through intermediate ‘linking blocks’. The homologous and linking blocks are aligned under PIP as independent DP sub-matrices and their tracebacks merged to yield the final alignment. The new algorithm can largely profit from parallel computing, yielding a theoretical speed-up estimated to be pro- portional to the cubic power of the number of sub-blocks in the DP matrices. We compare the new method to the original PIP approach and demonstrate it on real data

Can stochastic resonance explain recurrence of Grand Minima?

The amplitude of the 11 yr solar cycle is well known to be subject to long-term modulation, including sustained periods of very low activity known as Grand Minima. Stable long-period cycles found in proxies of solar activity have given new momentum to the debate about a possible influence of the tiny planetary tidal forcing. Here, we study the solar cycle by means of a simple zero-dimensional dynamo model, which includes a delay caused by meridional circulation as well as a quenching of the α-effect at toroidal magnetic fields exceeding an upper threshold. Fitting this model to the sunspot record, we find a set of parameters close to the bifurcation point at which two stable oscillatory modes emerge. One mode is a limit cycle resembling normal solar activity including a characteristic kink in the decaying limb of the cycle. The other mode is a weak sub-threshold cycle that could be interpreted as Grand Minimum activity. Adding noise to the model, we show that it exhibits Stochastic Resonance, which means that a weak external modulation can toss the dynamo back and forth between these two modes, whereby the periodicities of the modulation get strongly amplified

A comparison of numerical approaches for statistical inference with stochastic models

Due to our limited knowledge about complex environmental systems, our predictions of their behavior under different scenarios or decision alternatives are subject to considerable uncertainty. As this uncertainty can often be relevant for societal decisions, the consideration, quantification and communication of it is very important. Due to internal stochasticity, often poorly known influence factors, and only partly known mechanisms, in many cases, a stochastic model is needed to get an adequate description of uncertainty. As this implies the need to infer constant parameters, as well as the time-course of stochastic model states, a very high-dimensional inference problem for model calibration has to be solved. This is very challenging from a methodological and a numerical perspective. To illustrate aspects of this problem and show options to successfully tackle it, we compare three numerical approaches: Hamiltonian Monte Carlo, Particle Markov Chain Monte Carlo, and Conditional Ornstein-Uhlenbeck Sampling. As a case study, we select the analysis of hydrological data with a stochastic hydrological model. We conclude that the performance of the investigated techniques is comparable for the analyzed system, and that also generality and practical considerations may be taken into account to guide the choice of which technique is more appropriate for a particular application

High-density EEG power topography and connectivity during confusional arousal.

Confusional arousal is the milder expression of a family of disorders known as Disorders of Arousal (DOA) from non-REM sleep. These disorders are characterized by recurrent abnormal behaviors that occur in a state of reduced awareness for the external environment. Despite frequent amnesia for the nocturnal events, when actively probed, patients are able to report vivid hallucinatory/dream-like mental imagery. Traditional (low-density) scalp and stereo-electroencephalographic (EEG) recordings previously showed a pathological admixture of slow oscillations typical of NREM sleep and wake-like fast-mixed frequencies during these phenomena. However, our knowledge about the specific neural EEG dynamics over the entire brain is limited. We collected 2 consecutive in-laboratory sleep recordings using high-density (hd)-EEG (256 vertex-referenced geodesic system) coupled with standard video-polysomnography (v-PSG) from a 12-year-old drug-naïve and otherwise healthy child with a long-lasting history of sleepwalking. Source power topography and functional connectivity were computed during 20 selected confusional arousal episodes (from -6 to +18 sec after motor onset), and during baseline slow wave sleep preceding each episode (from - 3 to -2 min before onset). We found a widespread increase in slow wave activity (SWA) theta, alpha, beta, gamma power, associated with a parallel decrease in the sigma range during behavioral episodes compared to baseline sleep. Bilateral Broadman area 7 and right Broadman areas 39 and 40 were relatively spared by the massive increase in SWA power. Functional SWA connectivity analysis revealed a drastic increase in the number and complexity of connections from baseline sleep to full-blown episodes, that mainly involved an increased out-flow from bilateral fronto-medial prefrontal cortex and left temporal lobe to other cortical regions. These effects could be appreciated in the 6 sec window preceding behavioral onset. Overall, our results support the idea that DOA are the expression of peculiar brain states, compatible with a partial re-emergence of consciousness

Learning summary statistics for Bayesian inference with autoencoders

For stochastic models with intractable likelihood functions, approximate Bayesian computation offers a way of approximating the true posterior through repeated comparisons of observations with simulated model outputs in terms of a small set of summary statistics. These statistics need to retain the information that is relevant for constraining the parameters but cancel out the noise. They can thus be seen as thermodynamic state variables, for general stochastic models. For many scientific applications, we need strictly more summary statistics than model parameters to reach a satisfactory approximation of the posterior. Therefore, we propose to use a latent representation of deep neural networks based on Autoencoders as summary statistics. To create an incentive for the encoder to encode all the parameter-related information but not the noise, we give the decoder access to explicit or implicit information on the noise that has been used to generate the training data. We validate the approach empirically on two types of stochastic models

Extending Timescales and Narrowing Linewidths in NMR

Among the different fields of research in nuclear magnetic resonance (NMR) which are currently investigated in the Laboratory of Biomolecular Magnetic Resonance (LRMB), two subjects that are closely related to each other are presented in this article. On the one hand, we show how to populate long-lived states (LLS) that have long lifetimes T_LLS which allow one to go beyond the usual limits imposed by the longitudinal relaxation time T_1. This makes it possible to extend NMR experiments to longer time-scales. As an application, we demonstrate the extension of the timescale of diffusion measurements by NMR spectroscopy. On the other hand, we review our work on long-lived coherences (LLC), a particular type of coherence between two spin states that oscillates with the frequency of the scalar coupling constant J_IS and decays with a time constant T_LLC. Again, this time constant T_LLC can be much longer than the transverse relaxation time T_2. By extending the coherence lifetimes, we can narrow the linewidths to an unprecedented extent. J-couplings and residual dipolar couplings (RDCs) in weakly-oriented phases can be measured with the highest precision